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ABSTRACT 


Optimization problems involving linear systems with retardations 
in the controls are studied m a systematic way. Some physical 
motivation for the problems is discussed. The topics covered are! 
controllability, existence and uniqueness of the optimal control, 
sufficient conditions, techniques of synthesis, dynamic programming. 
A number of solved examples are presented. 
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1. Introduction 

Optimal control problems involving systems with delays in the 
state variables have been studied extensively and the difficulties 
encountered in such problems have been well documented [1, 2, 8, 15 , 
17, 25, 24, 27 and the bibliographies of 2, 24] . Recently, more 
sophisticated models with systems containing retardations m both 
the state and control variables have come under investigation [2, 4, 

6 , 7 , 12, 14, 17, 24] . In [2] Banks and Jacobs presented the mathe- 
matical foundations necessary for the study of very general control 
systems modeled by equations of the type 

o t 

x(t) = / x(t+s)d s F(t,s) + / h(u(s),s)d g G(t,s) 

-T -T 

where F and G are Stieltjes measures. The purpose of this paper 
is to investigate the effect (from both the theoretical and 
computational points of view) of lags m the control variables. We 
shall attempt to do this m a number of ways, but our aim always 
will be to point out the pathological differences between systems 
with delayed controls and those without. In order to isolate the 
effect of delays m the controls, we shall consider only the simplest 
models with lags in controls, and ignore any retardations in the 
state variables. Indeed, the examples of section 7 below illustrate 
very well the novel behavior of solutions to optimal control problems 
with these types of models. 

In section 2, motivated by models arising m current applications, 



2 


we formulate several different types of systems which appear to he 
of interest. Controllability of these systems is considered in 
section 3 where results involving controllability matrices analogous 
to those for non-delay linear systems are presented. In the next two 
sections the questions Q f existence, uniqueness, and sufficiency 
conditions for time optimal problems are considered in the spirit of 
[11]. In section 6 we extend to our systems a synthesis technique 
due to Neustadt [21] . A number of solved examples are presented m 
section 7 . These fundamental examples, governed by systems which at 
time t depend on the control at times t and t - h, are intrinsically 
more complicated than those involving systems which at time t depend 
on the control only at time t - h and give rise to prediction 
problems. Finally, the paper is concluded with a section concerning 
the applicability of dynamic programming techniques to certain cases 
of the systems under study, including mention of a Riccati type theory 
for quadratic payoff problems. 

We have tried to present numerous examples throughout the paper 
m order to provide the reader with an insight m regard to limitations 


of our results. 
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2. Notation and Formulation of Problems 

We shall denote by ;C the real vector space of all p X q. 
matrices. If A e ;£ the transpose of A will he denoted by A*. 

We shall not distinguish a column vector from its form as a row 
vector since it will always be transparent which form is intended 
by the order of multiplication m any matrix operations,, 

In order to facilitate the discussion of several types of problems 
involving various different system equations some special notation 
is required,, We denote by S^( A^B^B^) the system 


x(t) = Ax(t) + B Q u(t) + B 1 u(t-h) 


where A e g and h is a positive constant. The 

system 


x - Ax(t) + Bw(t) 

is denoted by S^(A,B) where A e £ , and B e £ . 

The term control means a triple {u^t^t^} where ul [t Q -h ; t^] 
-> R 111 is a function and t^t^ are real numbers. 

Ql 1 

Definition 2.1 . Given U C R the symbol denotes 

the system ^(A^ B^ ; B^) with constraint 


u(t) e U, 


t e [t^tj 
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on the controls {u,!^,!^}, t^t^ 6 R * 

Definition 2.2 , Given Uc/ and a bounded measurable function 

2 

v Q : [-h,0] ->U, we use i^ h (A, B Q ^B 1 ) to denote the system 
J9^(A, B 0 ,B 1 ) with constraints 


u(t) e U, 



* e CV 5 !^ 


on the controls, {u,tQ,t^), t^, t^ e R where 
s e [-h,0]„ 


u t (s) = u(t+s). 


Definition 2,3 , Given U C R m and bounded measurable functions 
v i : [-h,0] -> U, l = 0,1, we denote by A,B Q ,B 1 ) the system 
-9 ^ (A, Bq, B^) subject to constraints 


u(t) e U, t e [t^t-j-h] 


u t Q " V U t ] _ " V 1 


on the controls {u,t ,^5, t t e R. 

In the problems considered m this paper we shall often take 
U = R m or U - id 31 , where K 331 is the unit cube , 

{u = (u 1 , .„/) e R m lu 1 ! s 1, i = 1, m R m . Whenever, 

h, A, B , B^ are understood 5/^ will be used instead of 

-^h B q, B f) , l - 1,2, 3o 
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Systems of type Sr with v Q = 0 are models for physical 

problems where at initial time t Q there is no delayed control 

effect, hut after some time t^ + h there enters a non-negligihle 

effect on the system at time t by the control given previously at 

time t - h» This is exactly the case which occurs in the study of 

lossless transmission lines when one reduces a linear hyperbolic 

partial differential equation system with boundary controls to a 

linear differential-difference equation of neutral type m which 

control terms u(t), u(t-h) also appear linearly [l4]. 

Day and Hsia [7] have recently proposed a modification involving 

delayed controls for a model [18] of a gas-pressurized bipropellent 

2 

rocket engine. In addition to being of type this modified 

model also provides motivation for study of systems m which the 


kernel of B^ and the kernel of are complementary subspaces. 


5 ^- 


type systems are also models for continuous stirred tank reactors 


as studied by Ray and Solxman [24] . Although the example studied m 

[24] is non-linear, linearization about a nominal yields a system 

which satisfies kernel (b^C kernel (B^) (see section 4 below). 

3 

Problems with systems of type S/ with v Q = v^ = 0 are 
motivated by air traffic control models currently under study [26]; 
one such model has system equations x(t) = -,0(t,x(t)) + u(t-h), 
y(t) = q(t) - u(t), where A is a landing rate, q is a quemg or 
scheduling rate, and u is a takeoff rate. These models also in- 
volve systems m which kernel B Q and kernel B^ are complementary 

3 

subspaces „ Systems of type S/' with v^ = 0 are of importance m 
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so-called "settling problems"; i.e., problems m which one desires 
to attain the equilibrium state x (t^ ) = 0 m such a way that the 
system will remain at this state without further control if other 
disturbances are absent* 

We note that all three types of systems defined above are quite 
different from systems such as those modeling remote earth control 
of deep-space satellites studied by Foerster [ 9 ] and others [10, 

12 , 23 ] which contain only control terms with a delay (i.e., B Q = 0 ). 

A control {u,t Q ,t^} will be called admissible for the system 
S^( A,B^,B 1 ) (or simply S^ 1 - admi s s lble ) if u° [tQ-h,^] -4 R m is 
bounded, measurable and satisfies the constraints detailed m the 
definition of .S^ 1 , 1 = 1,2,3. Given x Q ,x^ e R n and t Q e R, we 
shall denote by P^, 1 = 1, 2^3> the problem of finding an 
admissible triple (u, t^,^} with U = satisfying x(t,° t Q ,x Q ,u) 

- x^ and TT = mm (t-J (u,t ,^3 is admissible with 

x(t-^; t 0 ,x 0 ,u) = X x }, where x(*; t^x^u) is the response (solution) 
of system S£( A,B Q ,B 1 ) to control u with x(t Q ; t 0 ,x Q ,u) = x QO 
That is, P^ denotes the time optimal problem from x Q to x^ for 
the system A,B Q ,B 1 ) with U = K 01 . The special case of problem 

with v Q = Y 1 = 0 will be denoted by P°. Finally, we shall 
denote by P the special time optimal problem as studied m [11] * 
i.e., the minimum time to origin for the system J/^A, B) with 
U- 

Necessary conditions m the form of a maximum principle for the 
problem P 1 are a special case of the general necessary conditions 
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derived previously lay the authors [2], Using similar proofs one 
can derive necessary conditions for the problems Pg and P^(P°). 
Use of these conditions yields that an optimal control {u^t^t} 
for problem must satisfy 


(2.1a) 


u(t) = 


sgn [t(t+h)B 1 ], t e [tg-hjt-h] 
arbitrary, t e (t-h,t Q ) 
sgn [i|r(t)B 0 ], t e [t Q ,t], 


if 0 S t - t 0 < h, and if h ^ t _ t Q , then {u,t Q ,t} must satisfy 


(2„lb) 


u(t) 


sgn [t(t+h)B 1 ], t e [t o -h,t Q ) 

sgn [i|r(t)B 0 + t(t+h)B 1 ], t e [t Q ,t-h) 

sgn [T|r(t)B 0 ], t e [t-h,t]. 


where jr(t) = tj exp(t-t)A, and the vector rj ^ 0 is an outward 
normal to a support hyperplane for the attainable set at tune t 
passing through the boundary point x^. It is understood that when 
a,b e R m , the relation "a = sgn b" is to be interpreted using the 
same convention as in [11, pg. 50], For the problem Pg one obtains 
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the corresponding necessary conditions from (2.1a) and (2.11) by 
deleting the requirements in the first two lines on the right-hand 
side of (2.1a), and the condition on the interval [t^-h,t^) m 
(2.1b). For problem (p ) one always has h so that the 

situation in (2.1a) never occurs. Thus the necessary conditions 
for problem P are obtained from (2,1b) by deleting the require- 
ments on the intervals [t^-h,^) and [t~h,t]. 

Any admissible control in problem satisfying the above 

necessary conditions for P will be called an extremal control 
for problem P , i = 1,2,3. Evidently, when computing extremal 
responses (i.e., responses to extremal controls) what one uses is 
what might be termed an extremal "effective control", i.e., v(t) = 
B Q u(t) + B-^u(t-h), t e [t Q ,t] where {u,t Q ,t} is an extremal- 
control. This v is easily computed from (2.1a) and (2.1b) or 
their appropriate modifications for problems P_,P„. 

2 3 



5 o C ontr ollab ility 


In this section we shall derive necessary- and sufficient conditions 
for controllability of the systems J/ 5 ’ 1 as defined above, These 
conditions will be analogous to the well-known rank condition on the 
controllability matrix for systems . 

Definition 3.1 . The system 1!>^) , 1 = 1,2,3, is controllable 

on [t_,t, ] if for every X..X, e R n there is an J^-admissible 
J- O* X 

triple (u,t Q , t.^ such that xft^ t ,x u) = x^. 

Remark 3.1 ° We shall find that the necessary and sufficient con- 
ditions for controllability are actually independent of the interval 
rvy as long as t^ > t^ + ho Hence one could define the 
equivalent concept of a "controllable system" in addition to a 
"controllable on [t ,^3 system". For the systems _S^(A,B) it is 
well-known that these concepts (and others) are equivalent [ 11 , 19 ]. 
Since we are mainly interested m obtaining the form of the necessary 
and sufficient conditions, we shall not pursue that aspect of the 
development here. 

Let us denote by Sgl £ x £ -» £ , x the usual controllability 

nn nr n(nr) a 

matrix B) x = [ B, AB, . . . , A n ~ 1 B ] . 

Theorem 3,1 . A necessary condition that j/^(A, B q,B^), l = 1,2,3 
be controllable on any [Iq, t^] with t^ > t^ + h is that 
[5f(A,B Q ), So{ A,B 1 )] have rank n. 
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Proof s controllable =>^(A, (BqjB^^)) controllable => 

$g{ A, (B^B^) las rank n => []^(A,B 0 ), ^(A,B )] has rank n. 


The above condition will be shown sufficient for systems 

and Sf whenever U = R } but a much stronger condition will be 

?3 


necessary and sufficient for systems jy 5 ’" 
does not depend on h, the lag size. 


Note that the condition 


Theorem 3.2 „ Let U = R m „ A sufficient condition that 
,S^(A,Bq,B^) and .S''^(A,Bq,B^) be controllable on every [t ^t.^] 
with t^ > t + h is that [^(A^B^), JjgffAjB^)] have rank n. 

p 

Proof a It suffices to give the proof for the system ^ (A, 

We shall give a proof that is a slight modification of that given 
for the systems jy^A^B) m [19] . The usual constructive proof 
(see [13]) using a special symmetric matrix can also be made. Assume 
that [^(A,B Q ), ^ 7 (A j B 1 )] has rank n. Let [t t.^ with 
t-j_ > t Q + h and v Q be given for 5 ^( A j B 0 , Bp) . Define x Q (v 0 ) = 

-(t r t )At° (t -t-h)A 

-e / e b,v (t-t )dt and consider jar (x (v )), the 

* J- O U 0 U 

V 

2 

attainable set at time t^ for the system ^(AjB , B^) with 

x(t 0 ) = x 0 (v 0 ) and U = [u e R m | | u X | g M, i = l,...,m}. The set 
M 

( x q^ v o^ consists of all points z of the form z = 

V h (t -t-h)A t l (t -t)A 

/ e B u(t)dt + / e B u(t)dt where uj [l.t ]->E m 

4- ■*- 4. U 0 7 -L 
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is bounded measurable with j u^t) j g M. We claim that _£)/^(Xq(Vq) ) 
C R n has dimension n. If not, there is a vector X ^ 0 such that 
Xz — 0 for all z e (Xq(Vq) ), or 

t l" h (t -t-h)A t l (t -t)A 

(3.1) 1 / e B,u(t)dt + X / e B n u(t)dt = 0 

t 0 t o 

for all bounded measurable u with |u 1 (t)| g M. Taking u = 0 

(t -t)A. 

on [t^t^-h] m (3<»1) yields Xe = 0 for t e [t^~h,t^] 0 

It follows by the usual arguments that IA^Bq = 0 for 

k = 0,1,2,...* thus x5£(A, Bq) = 0 and le^Q = 0 for | e R. 

(t -t-h)A 

Use of this latter result with (3°l) yields Xe = 0 for 

t e [t Q ,t 1 -h]. It then follows that X[ _^(A, B Q ), A,B 1 )) - 0, 

contradicting the rank condition hypothesized above. 

That the n-dimensional set (v q ) ) is compact and convex 

n 

m R follows from previous results by the authors [2] . Further- 

M 

more, it is easily seen that (x (v )) is symmetric about the 

0 0 

origin in R n and hence must contain a neighborhood of the origin. 
m 2M 

Since 2j# (Xq(v q ) ) cj/ (x 0 (v Q )) we find "that the attainable set 

2 -j. 

J 2 ^(x 0 (v 0 )) at time t x for S/ with U = R and x(t Q ) = x Q (v 0 ) 
must be all of R n . The conclusion of the theorem then follows from 
the fact that 

{V t o^ A 

_£/(x 0 ) = e {x 0 -x 0 (v 0 )) + j/(x Q (v Q )) 

for any x Q e R n . 
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We remark that an obvious modification of the above proof will 

show that the condition of the theorem is also sufficient for con- 

2 

trollability of systems of type (A,Bq, B^) where one has a 

boundary condition' u , = v in place of u = v . As one would 

t l X ^0 ° 

expect, if U is a proper subset of S , then the condition of 
Theorem 3.2 is no longer sufficient for controllability (see examples 
7.3, 7*^ below). An immediate consequence of Theorem 3«2 is that 
systems Lx = b u(t) + b^uft-h) will always give rise to and 

S/ type systems which are controllable „ Here L denotes the 
usual real scalar n^ order differential operator with constant co- 
efficients, Lx = x^ n ) + a n x^ n_1 ^ + ... + ax, 

’ n-1 0 


Remark 3.2 . In a recent note [5] D. H. Chyung considered the con- 

2 

trollability question for systems of type S / 7 . He obtained as 
necessary and sufficient for controllability the condition that 
[if(A,B 0 ), SfCA^ -1 ^)) be of rank n„ Rote that from this con- 
dition one might suspect that lag size h could affect controllability. 
However, it is not difficult to show that [^(A,B ), if( A,e“ hA B 1 )] 
has rank n if and only if [^(A,B ), ^ 7 (A,B 1 )] has rank n. From 
a practical, point of view, use of the second matrix is more desirable 
since it can be computed without computing e -1 ^. 
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In practice when delays are small xn a problem one sometimes 
chooses to ignore them and work with an approximate system obtained 
by setting h = 0 m the original system. For x = 1,2, the sys- 
tem 5^ (A, Bq,B^) is thus approximated by the system 5^(A,B 0 +B 1 ) o 
In connection with this approximation we make the following observa- 
tion. 

Theorem 3.3 . For l = 1,2, ^’(A,B 0 +B 1 ) controllable implies 

j^(A, B q _, B^) controllable when U = R m „ 

Proof s S/{ A,B Q +B 1 ) controllable => ^’(A,B 0 +B 1 ) has rank n => 
[B 0 +B 1 ,A(B 0 +B 1 ), . . -B q , -AB q , - . . has 

rank n =|> [j^(A,B 0 ), $£(. A,B 1 )] has rank n. 

It is easy to give an example to show that the converse of 
Theorem 3=3 is not true, e.g„, take = -B^. Indeed, even xn 
situations where the approximation might seem more reasonable, con- 
trollability can still he lost by use of the approximation. 

Example 3 .1 . Consider the system 

x(t) =s fy(t) + au(t) + hu(t-h) 
y(t) = gx(t) + cu(t-h) 


where a, b, c,f, g,h are not zero. One finds that 



[#(A,B ), ^ 7 (A,B 1 )] has rank 2 while det A ,B () +B 1 ) = 

2 2 

g(a+b) - fc e Thus by using the approximation one destroys con- 

2 2 

trollability if (a+b) = fc /g. For example,, if a = 1, h = -5 

and c = (g/f) 1 / 2 (1-g) where g/f > 0 ; one would probably not wish 
to ignore tbe lag h„ 

We remark that the results of this section can be extended to 
systems with multiple delays and even to systems with certain types 
of time variable delays. For example^ for systems with dynamics given by 

V 

(3.2) x(t) = Ax(t) + S B i u(t-h i ) t e [*0,^3 

1=0 

with 0 = h Q < h x < < h^ and u(t) e U, t e [tg-h t-J, one 

can modify the previous proof to obtain the following theorem. 

Theorem 3.^ . Let U = R m . A necessary and sufficient condition that 
(3.2) be controllable on any [t ,t_^] with t^ > t^ + h^ is that 
[#(A,B q ), ^(A,B 1 ),...,i^(A,B v )] have rank n. 

As a corollary to this theorem we obtain a sufficient condition 
for controllability which does not involve A. 

Corollary 3d . For the system (3.2) with U = R m and (-\H-l)m § n, 

a sufficient condition for controllability on any [t 1; ] with 

t n > t. + h, is that ] have rank n. 

1 0 V L Cr l* * V 
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Once one has obtained necessary and sufficient conditions for 

1 2 

controllability of systems B^B^) and m 

terms of a rank condition on a "controllability matrix " } one should 
be able to prove many theorems for these systems analogous to those 
for the system which involve the usual controllability 

matrix ^ We shall present one such result involving the domain of 
null controllability, the proof being developed m a manner similar 
to one m [19]. 

We define the domain of null controllability for 



X o €K 


n 


there exists an 

K w with x (V V 


L y -admissible triple 
= 0 [ • 


In a similar manner we define for the set 


%(\) S 


x 0 e R' 


n 


there exists an S/ -admissible triple 


{u,t^, t^} with u^_ = v such that 

*0 


x (\; = ° 


O T 

Wote that for a given U, ^ Q (v Q ) C for any v Q „ We shall be 

£ 

especially interested in the set £^(0), i.e„, v Q =0. 



1 6 


Lemma 3.1. Suppose 0 e U C R and A is asymptotically stable. 
If ^(0) contains a neighborhood of the origin m R n , then 

^( 0 ) = R n . 


Proof : Given x e R n , let T > 0 be such that x(t; e x ( 


0 




is m JY C ^ (0). Then let {u^t^ t^} be J/'’ -admissible 


with 


tA 


u + = 0 such that xft-J t ,e x^u) = 0. Defining 


“0 


'O’ 


"(I) = 


u(i-T+t n ) 


i e OM] 


| e (T,T+t r t 0 ], 


it is easy to show that x^+t^-t^J O^x^u) = 0 which implies 

*o s OJ l 


Lemma 3.2 . Suppose 0 e int U and [^(A^B^), .^(A^B^)] has 

2 n 

rank n. Then i^(0) contains a neighborhood of the origin m R , 


Proof! Let S& (y ) denote the attainable set at time t.. 

* 0 X 

spondmg to y(0) = y Q using the system 


r 

y(t) = -Ay(t) - B x w(t) - B Q w(t-h) t e [0,^] 

w t = 0 ; w(t) e U, t e [-h,^]. 


corre- 
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2 

This system may be thought of as the system with 

Vq = 0" run m reverse time. Since rank -B^) ? 5g(-A } ~Bq) ] 

= rank [_£f(A,B Q ), .^(A^B^)], arguments similar to those m the 

proof of Theorem 5.2 may be used to show that _Q/~ (0) contains a 

t l 

neighborhood of the origin m R for t^ > h whenever 0 e mt U. 

It remains only to show (0) C (0) for t > h. Since 

x, e jaC (0) is of the form 
i t x 

h (V S H- A > 

X- = / e [-B,w(s) - B w(s-h)]ds 

0 xu 

where w, =0, one can easily obtain 

ti ; 

t.A \ (t.-t)A 

0 = e x + / e [B u(t) + B u(t-h)]dt 

x 0 0 X 

with u(t) = w^-h-t) for t e [-h,^], yielding that x 1 e ^^(0). 

Combining the two lemmas one obtains the following results. 

Theorem Suppose A is asymptotically stable, 0 e mt U, and 

[ (A, Bq ) , Sf (A, B^) ] has rank n. Then i^(0) (and hence 
is all of R n 0 

Obvious modifications of the above arguments yield the following 


corollary. 
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Corollary 5»2 » Under the hypotheses of Theorem. 3.5, we have 
2 rj 

9 0 (* 0 ) = R for 21117 V 

Remark 3.3 . One can obtain a result similar to Theorem 3.5 for the 
systems Sr ^(. with the condition - v Q replaced hy 

= v 1 » However, the rank condition of the hypotheses must be 

replaced by the, m general, stronger condition " ^(A, ) 

has rank n". The reason for this change will be apparent after our 
discussion on the controllability of systems of type 5^^ (A, B^B^) 
which follows. 


Although controllability conditions for systems 3^) 

can be derived from basic principles as was done above for systems 
1 2 

S/ and Sf , we shall make a simple observation about systems of 
type S/ which will yield the same results immediately by applying 
known theorems [11, 19] for certain non-delayed systems. For 

on C t 0 ^ t i] an-* 3 - v v x given, a straightforward 
calculation shows that the response x(-; t 0 ,x Q ,u) to -^^( A ^ B 0 ; B i) 
satisfies 


where x 
subject to 


x( V V x cy u > = vv u) 


is the solution to system S^(A } e^B +B ) 


xft ] - x - 4. ~ ( t i- h - t 0 ) A 

x(t Q ) - x - e x + e 


0 l y 
A with 


°“ [to.*!- 11 ] 

A - 
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defined by 

*0 (t -t-h)A 

s l „ e B i v o (t - t o )dt 

V 

t l (t -t)A 

+ / e B V (t-t )dt. 

V h 

There fore ^ it is not difficult to verify that B^) is 

controllable on if and only if S/%A } e hA B Q +B 1 ) is con- 

trollable on [t^^t^-h]. It follows that studying controllability 
of systems is equivalent to studying that of systems 

e^B^+E^) . Since the matrix e^B^+B^) is rank equivalent 

to ^’(A,B Q +e'' M B 1 ), we have the following theorems. 

Theorem 3 «6 . A necessary condition that j/^A^B^B^) be con ~ 
trollable on any [t^t-J with ^ > t Q + h is that ^’(A,B 0 +e" hA B 1 ) 
have rank n. 

Theorem 3.7 ° Let U = R m . A sufficient condition that 

Sf h ( A J B QJ B 1 ) be controllable on any [t^^] with > t + h is 

that #(A,B ) have rank n. 

Remark 5A , The rank of j^(A,B 0 +e~ M B 1 ) equals n implies the 
rank of [^(A^B^), ^(A^B^)] is n } but not conversely (see 
Example 3„2 below). Thus the rank condition of Theorems 5.6 and 5.7 
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is, m general, stronger than that of Theorem 3.1. Furthermore, 
the dependence of the rank condition here on the lag size h is 
not illusory (see Remark 3.2) as the following example demonstrates. 

Example 3.2 . Consider the system 

x(t) = Try(t) 

y(t) = -Tix (t) + u(t) + u(t-h) . 

For h = 1 we find if (A ,B +e" hA B 1 ) = 0 while for h = 2 

. In addition, [^(A,B q ), j^XA^)] 

has rank 2. 

The above example also shows that the systems Lx = b u(t) + 
b^u(t-h), u_ t = v^, u^ = v^, need not be controllable (see the 

comments preceding Remark 3.2). It is also easy to see that con- 
trollability of ,S^( A,B Q ,B^) is not, m general, implied by 
controllability of either S^(A } B^) or S^(A, B^) . 

That a result on approximation similar to Theorem 3.3 does not 
hold for ^3 

type systems can be seen from Example 3.2 above. 
Finally, defining the domain of null controllability ^q( y q} y ^) 
for ^(A,B 0 ,B 1 ) m the obvious way, we do obtain the following 


^(A,B 0 +e“ hA B 1 ) = 


'0 2 TT 

2 0 


analogue to Theorem 3.5. 
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Theorem g,8 o Suppose A is asymptotically stable, 0 e int U, 
^(A,B 0 +e" hA B 1 ) has rank n„ Then, ®o<V T i> = R n for any 


and 
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4. Sufficient Conditions for the Special Time Optimal Control Problem 
In this section we prove sufficient conditions for problems of the 
form P^, or P° where U = K** 1 , the "unit cube" m R m (see 
section 2) and the terminal condition x(t-^j t ; x ,u) = 0. Actually, 
m sections h through 7 we always take t = 0 so t Q will he 
suppressed m the notation x(t° t^x^u) and m the notation 
[u, t ,^1 for an admissible triple. The sufficiency condition m 
this section is an extension of a sufficient condition given by Hermes 
and LaSalle [11, pg. 72], The discussion is facilitated by introducing 
the concept of the set of reachable states at time t [11] for 
problems P, P^, P^, and P°„ We say that a point (or state) x e R n 
is reachable at time t s 0 m problem P if there is an admissible 
u for problem P such that 

t Ag 

(Ij-.l) x = / e Bu(s)ds. 

0 

We say that x is reachable at time t § 0 m problem P-, , P 0 , P° 

■ ■ 1 ■ ■ — ■■■ JL * 7 d? $ 

if there is an admissible {u, t] for problem P^, P £ , P° respectively 

such that 


, -As 

(^-.2) x = / e [BqU(s) + B 1 u(s-h)]ds. 

The symbols ^(t), ^(t), ^? 2 (t), ^°(t) denote respectively the 
set of all states x reachable at time t m problems P, P^, P 2 , P°. 
Properties of <%(t) have been carefully studied m [11]. The 
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■behavior of i^(t) and i^(t) is somewhat more complicated. In 
fact, we shall see that some of the basic properties of &(t) 
simply are not true for -^(t) and ^(t) without making special 
assumptions on B Q and B^o 

If x,y e E P , then we use <x,y> to denote the usual scalar 
product in R P . If SC R P , then S 1 denotes the orthogonal comple- 
ment of S, i.e (J = {x e R P j <x,y> = 0, y e S]„ If M is a 

p x q. real matrix, i.e., M e f , then we reserve ker (M) and 

PQ. 

Im (M) for the kernel and image respectively of the linear trans- 
formation x H t-xM, x e R P , i.e., ker (M) = (x e R P | xM = 0} and 
Im (M) = (y e E^| y = xM for some x e R P }o The following norms 
will be used for vectors x = (x 1 , ...^x 15 ) e R P S 

INI* = max (lx 1 !, i = l,...,p} 

INI = 

|xj = Z lx 1 ] . 

1-1 

We also use the symbol ||m|] ^ to denote the matrix norm subordinate 
to the vector norm j| ■> || on R p and R^, l . e . , 

l|M|| M = max (||xM|[J INI* SI, x e R P } 

P 

= max f Z Im j 
i=l 


2 — ^ l] * 



2h 


where M = (m ), i = 1, .■><>, p; j = 1 , ...,q. The matrix norm |m| 
subordinate to the vector norm j •] is similarly defined and is 
likewise easy to compute. 

game hypotheses which will be invoked to obtain various results 
m the sequel are now listed for future reference. 


(HI) 

Matrix B_ has a left inverse B ^ and 

0 UJj 

C = B~^B, satisfies |[c|| g 1. 

OL 1 " 11 oo 


(H2) 

Hypothesis (Hi) with jc| < 1 instead of 

HclL g 1 

(H5) 

For 

any t-^tg satisfying 0 < t_ L < t £ 




\ t 2 
/ | r|e _AS B 0 l ds > / j qe“ As B | ds 

*1 \ 



whenever t| e R n and tj / 0, 


W 

No 

1 > IxBj, x e [ker (B^] 1 = Im [B*], 

x/0, 

(h4>) 

No 

| g jxB 1 |, x e R n . 


Proposition 

4o1d 

There is an m X m real matrix G 

such that 


B 1 = B^G if and only if ker (B q ) C ker (B 1 ). 


Proof " Evidently, B 1 = B Q G implies ker (B q ) C ker (B^) . Conversely, 
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ker (B q ) C ker (B 1 ) implies [ker (Bq)] 1 3 [ker (B^) ]' L or 
equivalent ly, Im (B*) D Im (B*) » The existence of G with the re- 
quired properties follows at once from the last inclusion and some 
elementary matrix operations. 

Proposition 4.2 . (a) (h 4 ! ) implies ker (Bq) C ker (B-^). 

(b) (h 4) and ker (Bq) C ker (B^) imply (H4- 1 ) . 

(c) If _9 %A.jBq) is proper 1 , ker (B q ) C ker and (H4-) 

is satisfied, then (H5) is satisfied, 

(d) If (H3) is satisfied, then A,B Q ) is proper, and (h 4' ) 
is satisfied, 

(e) (HV ) and (H2) imply (h4) . 

Proof" Statements (a) and (b) are obvious. Suppose (H3) is 
satisfied. Then for 8 > 0, T) e R n , rj / 0 we have 

t^+5 t^+8 

^ / | ne- As B 0 j ds > ^ / j T|e“ As B^| ds, > 0. 

t l t l 

Hence there results 

he~ AS B 0 l 5 1 T|e“ As B 1 | , r\ e R n , s 1 o 

and (h 4' ) is satisfied. Evidently (HJ) implies A,B q ) is proper, 
"'"See [11] for the definition of a proper system J^(A,Bq) . 
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Now assume (h 4), ker (B q ) C ker (B.^, and i^(A, B Q ) is proper. 
Observe that 

R n = ker (B q ) © [ker (Sq)] 1 = ker (B Q ) © Im (B*) . 

Choose rj ^ 0^ rj e R and define ijr(t) = B e « Then 4r(t) = 

v(t) + p(t), where v(t) e ker (B Q ) and p(t) e Im (Bq) • This de- 
composition is unique and q and v are continuous . Choose 
0 < t < tg, then (H4) implies 

| M-(b)B Q | > ld(t)B 1 | 

on with the possible exception of a finite number of 

points since i^(A,B 0 ) is proper. The assumptions ker (B q ) C 
ker (B^) implies 

I i!r(-fc)B Q l > it(b)B 1 i a.e. on [t^tg] 
and (S3) follows at once. 

Suppose now that (h 4' ) and (H2) are satisfied. Then B^ = B Q C 
by Proposition 4.1. If x e [ker B^] 1 , x ^ 0, then J | > 0. 

Whence jxB-J - I^q^I - |xB Q | j C [ < |xBq|, and (h 4) is satisfied. 

Corollary 4.1 . If SS(A, B Q ) is proper, ker (B q ) C ker (B ), and (h4) 
is satisfied, then [r|e“ At B 0 i > he^B-J, t] f 0, rj e R n for all but 
a finite number of t on any compact interval. 
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Example 4 . 1 . Let L denote the linear differential operator 


Ik - .W 


a 

n-1 


(n-1) 

' ' 4 - • • * 4 * 


a o x 


where a , i = 0, ...,n-l are real constants. Consider the control 
system Lx = "b u(t) + b^u^t-h) where b 0 ,b are real constants. 
Since we refer to this example several times m the sequel we write 
this explicitly m the form 5^(A ; B . Let 



Then Lx = b^uft) + b^u(t-h) is equivalent to the system 
^f(A,B B ). The condition that matrix B have a left inverse is 


0>1 

equivalent to b^ f 0. Hypothesis (Hi) is satisfied if h j 0 and 


*3 


g 1. Clearly ker (B Q ) C ker (B^) . Moreover, h f 0 implies 

S/{ A,B Q ) is proper. Finally (h 4) is satisfied if [b^] > |t» 1 j . 

, Let T(t) denote any one of the reachable sets at time t (i.e., 
^(t), i^(t), ^ 2 ^)} Then P(t) is increasing if 

0 ^ ^ S tg implies r(t ) C r(tg). We say P(t) is expanding if 
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P(t) C Int (r(t 1 )) for Oit< t 1 . Let K g denote the character- 
istic function of a set S C X. Define 


(4.3) u(s; t,Ti) = sgn [i!e“ AS B 0 K [0 ^ t:] (s) + ne" A ^ S+h ^ B- jKj. t _ h *j ( s ) 1 

for -h g s g t and i) eE“ r\ / 0. When u(s; t,r|) is referred 
to with -h g s g t it is understood that we are referring to 
problem P^» The corresponding u(s* t, r]) for problem merely 

requires u(s; t,q) have the foim (^.3) for 0 g s g t and 
u Q (-; t,i)) = In problem P° we do not invoke this symbol. The 

notation cp(t,u) where {u,t} is admissible in or is 

defined by 

t . 

cp(t,u) = / e” AS [B n u(s) + B 1 u(s-h)]ds. 

0 U 

It is also convenient to take the following definitions, 

(^•5) z(t,ri) = cp(t,u(«; t,T!)), 

(4,6) g(t,T]) = <T],z(t,Tl)>. 


Proposition P^(t) 


and (t) are increasing. 


Proofs The statement concerning i^(t) is obvious. Note that 
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3^°(t), t I h is merely ^(t-h) for problem p -with system 
^(A,B 0 +e -Ah B 1 ) . For 0 £ t £ h, &°(t) = [0], so ^°(t) is in- 
creasing,, 

Proposition 4*4 * .^g(t) are Both increasing if any of the 

following three conditions is satisfied^ 

(1) ker (B q ) C ker (B^) and (Hi) is satisfied* 

(2) ker (Bq) C ker (B^) and a matrix G satisfying the con- 
ditions of Proposition (k*l) also satisfies 1[g[| ^ S 1* 

(3) (H^' ) is satisfied* 

Proof * We shall prove only that & (t) is increasing is implied 
by (2) or (3) « The remaining situations are similar* If 
ker (Bq) C ker (B ^) } then an m X m matrix G exists satisfying 
= BqG (Proposition 4*1) 0 If (2) is true we may take ||g|| £ 1* 
If P = cp (-fc^n) and 0 < tg - < h, then w(t) = -Gu(t-h), 

t-^ h t S tg is measurable and satisfies 

ll'w(t)|| oa £ 1, 

B w(t) + B^u(t-h) = 0 

on [tptg] 0 Befine u^° [-h^tg] -» K 111 by the conditions 
^lIC-Mil = ^ ^d uj (t 1# tg] = w. Then cpftg,^) = cp(t^u) = p e 
-^(tg) and we infer that il^(t) is increasing. 

Suppose that (h 4 ! ) is satisfied* Choose t^ and tg such that 
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0 £ S tg and pick tj e R 1 ^ r] f 0. There are support hyperplanes 
ir^ and '^~2r\ ^ 1 ^ 2 ) res P ec f with outward 

normal r|„ Thus there exist e i^(t^) and p^ e such 

that Ci^p^ S <Tj,q>, q. e and <n,p 2 > i? <T],q>, q. e ^(tg) . 

Hence <n,p ;L > = sCt^ii) and Oj,p 2 > = g(t 2 ,Ti). How g(t,Tj) as 
defined in (^-.6) can be written m the form 

(^-7) g(M) = / he“ A ^ S+ll ^B 1 |ds + / 1 rie" As B + rie" A ^ S+h Vids 

-h 0 U 

^ , As , 

+ f |n e B n |ds, lf t - h> 

t-h u 

From (^oj) and (H^ 1 ) one deduces that ^ (t, T]) ^ 0, t I h and 
t | — > g(t^ Tj) ^ t S h is nondecreasing. Therefore, if S h, then 

H lr] = {q. e R n | <Tl,q> g gCt^Tj)) C = (q. e R* 1 ] <n,q> =S gCt^T))), 

and since ^ (t, ) = D H_ and & (t 0 ) = H H„ we have 

11 i; rfP* 1 2 V° ^ 

C ^(t 2 ) for t g * t ± 2 h. If 0 * t 1; t £ h, then 

^^(t.|) C is^(t 2 ) 1S cdear » The fact that i^(t) is increasing 

is now a simple deduction,, 

It is easy to construct examples that show that (1), (2), and 
( 3 ) m the preceding proposition are in general independent. We 
give below two examples showing that the conclusion of Proposition 
need not be true if some of the assumptions of Proposition 4.^ 
are dropped. 
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Example 4.2 , Consider the scalar control system, x = x + u(t) + 
Ku(t-l) where 1 - 2e < K < -e«, Using this system with problem P^ 
we see that 

g(t,l) = / |Kj e" ( ' S+1 '^ds + / je" S + Ke"^ S+1 ^|ds 
-1 0 

t 

+ / e~ s ds, t > 1. 
t-1 

Since i^ft) = [-g(t,l), g(t,l)] is a compact interval and since 
(t,l) < 0 for t > 1, it follows that &^(t) is not increasing,. 



problem P 2 with v Q = 0 o Then her (B q ) <£. ker (B^) and ^ 2 (t) 
is not increasing. For example, for t > 1 define p(t) to he 
max {x e r| (x, 0) e P^ 2 (t)} . Then 


t 

(i(t) = / e -s ds 
t-1 

so that p decreases for t > 1. 

Proposition 4<,3 o i^°(t) is expanding t?h if and only if 

✓ » 

is proper. 

Proof ” This follows at once from [11, pg„ 73] and the remark m the 
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proof of Proposition b.3° 

Proposition 4.6 „ if (H3) is satisfied, then ^ (t), i = 1,2, are both 
expandingo Moreover, if ker (Bq) C ker (B^) and i^(t) or ^^(t) 
is expanding, then A,B Q ) is proper. 

Proof. Note that ^(t), 1 = 1,2, are increasing (Propositions 4.2d 
and b.k)o Choose t-^tg satisfying 0 < ^ < tg. Pick qe ^(t) C 
^(t 2 ). If q ^ Int (^_(t 2 )), then q e Bdf^ftg)), the boundary 
of -^j_(t 2 ). Consequently, there is an r] ^ 0 which is an outward 
normal to a support hyperplane for i^(tg) through qj i.e., 

<r),p-q> ^ 0, p e ^(tg). 

The point q has the form. q = 9(t..,u) where {u,t^} is admissible 
m P l0 A function Ug° [-h,tg] -» is defined by 

u(t), -h g t g t 

«2 ( t) = 

sgn [qe“ A1S B 0 ], t 1 <tSt 2 

Then is admissible m P . If p = cp(tg,Ug), then 
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^ / [he- As B 0 | - he-^B-Jjds 
t l 

> 0 

"by (H3). This is a contradxctxon. Hence (t) is expandxng. 

The same proof works for . 

How suppose ker (B q ) C ker (B^) and il? (t) xs expanding,, 

If V(A,-B Q ) is not proper, then there xs an T] / 0, t] e R n such 

— A*t — A"b 

that Tje Bq = 0, and consequently qe B-^ = 0. Now the control 

function u^ = 0 has the form u(s’ t, t)) (see Equation 1 4-.3). 

Hence 0 e Bdf^(t), t > 0 so that J^(t) is not expanding. 

Analogous reasonxng holds for the case where i^-,(t) is expanding. 

It will be pointed out in sectxon 7 when some solved examples 

of problems of type p^, P^, P° are presented that hypothesxs (H3) 

cannot be omitted and still obtaxn (t), x = 1,2 are expanding. 

Indeed, as we shall point out xn the discus sxon of those examples, 

the hypotheses of the fxrst part of Proposxtxon b.6 cannot be 

weakened, and there does not appear to be an analog of Theorem 17 .2 

xn [11]. The sufficient conditxon of Hermes and LaSalle [11] can now 

be stated. 

Theorem 4.1 . Let r(t) be any one of the reachable sets at txme t, 
^(t), ^(t), ^ 2 (t), If r(t) xs expanding, if {u,t} is 

an extremal control for the corresponding problem, and xf 
x(t* Xq,u) = 0, then (u, t] xs a txme optimal solutxon to the problem 
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associated with r(t) . 

The proof of this theorem is obvious* Of course, the result is 
not of much interest without computable criteria for showing P(t) 
is expanding* Propositions 4.5 and 4*6 m conjunction with 
Propositions 4*2, 4*3, and 4.4 give us such criteria. 

and = (^ ^ ) • ^(A,Bq) is not proper, and it is easy to 

see ^(t) is expanding* Moreover, (H5) is not satisfied* 
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5. Existence and Uniqueness for the Tune Optimal Control Problem 
It is easy to modify the uniqueness theorem m [11., pg. 69 ] to 
apply to problems P^ 1 = 1,2,3 0 Two admissible controls 
{u^ti), {u^tg} are regarded as equivalent if t^ = t g and 
u^(t) = u^(t) a.e. on [-h,t ]. An admissible control fu^t^} for 
P^ is said to be bang-bang if [u(t)| = m a.e. on [-h,t^]. 

Similar ly, an admissible control {u,t^} for P^, (respectively 

P_) is bang-bang if the above condition is satisfied a.e. on 

5 

[Ojt^] (respectively [O^t^-h]). The following extension of a 
result m [11] is obtained,, 

Theorem 3°1 ° If (u,TJ} is an optimal solution to P implies 
{u,t} is bang-bang^ then there is at most one optimal control for 
problem P , 1 = 1,2,3° 

Proofs One merely supposes there are two optimal controls {u^T ;} , 
[Ug^t] m problem P_^ which differ on a subset of [-h^t] of 
positive measure,, Then x(t‘ x^v.^) = x^ = x(t° Xq^u^). If we 

m + 

define w l [-h^t] -*■ K by w(t) = - , then (w^t) is 

admissible m P » Moreover, it is clear that x(t° x„,w) = x_. , and 
(w,t) is not bang-bang. This is a contradiction. 

One never obtains uniqueness of the optimal control problem P n 
if 0 S t § h since the control (u, t} is not effective m 
^(A^BqjB-^) for t - h S t S 0. For this reason when we discuss 



36 


uniqueness of the solution to problem we assume t s h, This 

is only a manor point and the situation 0 £ t g h can essentially 
be treated as in [11] . 

The next result is a reformulation of a general existence 
theorem obtained in [2] . Actually, problem P was not discussed 
there, but the existence theorem easily extends to this situation. 

Theorem 3.2 . If there is at least one admissible control (u, t^} 
for problem P satisfying x(t_^° x^,u) = x^, then there is an 
optimal solution to problem P , i = 1,2,3. 

# 

Proposition 3<,3 o There is at most one solution to problem P if 

i^(A,B 0 ), ^(A^Q+e-^), and ^A,B 1 ) are normal (see [11] for 

the definition of normal). The statement of uniqueness holds for 

—All 

problem P^ if Sf (A, B Q + e ) and S^(A } B Q ) are normal, while 

for P, the normality of ^(A.B +e”'^ 1 B n ) suffices, 

3 0 1 

Proof , We consider only problem P . Clearly, the necessary con- 
dition (2.1) and normality of the three systems imply that the 
hypothesis of Theorem 5«1 is satisfied. 

+ The problems P^, i = 1,2,3 were formulated so that the admissible 
controls were in the class of Lebesgue measurable functions. The 
results m [2] when specialized to the present situation reveal 
that we could just as well have restricted our attention to piece- 
wise continuous controls. 
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-Ah 

If and e are known, then computable conditions 

assuring the normality of SS(A } Bq), -S^(A ; B^+e" B^), and -S^(A,B^) 

-Ah 

are given in [11] „ In general, e is difficult to determine so 
we would like to obtain conditions that can be directly computed 
from A; B^B^. (in this connection it should be observed that, m 
general, the normality of any two of the systems J9 ^(A,Bq), 

-9^(A, B^-re ^b^), S/(k, B^) does not imply the normality of the 

third. For instance m Example 5.2 J5 ^(A, Bq) and 5^(A, B^) are 

normal but ^’(A,BQ+e~^ :1 B-^) is not normal if h = 1.) Some results 
are possible in this direction,, For example, let us consider the 
control system S£( A,E q ,B 1 ) discussed m Example 7,1. Along with 
the differential operator L m that example we consider its ad- 

■X" 

joint L given by 

_* (n) (n-1) , ,,n 

L x = x v ' - a n x v ' + »«• + (-1) a„x, 
n-1 ' 7 0 

It is now assumed that |b^| + |b.J f 0 m Example 7.1. 

Proposition 5.7 . System 5 /# ’(A,B 0 ) is normal if and only if b Q ^ 0. 
System jF^A, B^) is normal if and only if b^ 0. If b = t^, 
and if Lx = 0 has no nontrivial solutions of period 2h, then 
i^(A,B 0 +e'' Ail B i ) is normal. On the other hand if b Q = -b^, then 
^(A^^e'^) is normal if and only if Lx = 0 has no nontrivial 
solutions of period h. 

Let 1(A) denote the eigenvalues of A, and let Re 1(A) denote 
the real parts of 1(A). 
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Proposition 5.5 . If | TOqI < 1 h^] , and if Re A.(A) ^ 0, then 
5^(A,B 0 +e'' Ah B i ) is normal. 

f 

Proposition 5„6 <, If |b Q ( > jb-J, and if Re A, (A) 1 o ? then 
) is normal. 

Propositions 5°5 and 5=6 are pretty clear, so we will only 
indicate the proof for one of these (proposition 5.5). If = 0 } 
then Proposition 5.5 is true. Thus suppose b^ ^ 0. Suppose 
5^(A,B 0 +e~ AIl B 1 ) is not normal. Then there is a nontrivial solution 
i{r of l/x - 0 such that 


b Q f(t) + b-^ft+h) = 0. 


An easy induction argument shows that 


( 5 . 1 ) 


f(t+Kh) = (-1) K 




t(t) 


K = 1,2,5,... . Since b (t) is nontrivial there is a sequence t 

K. 

such that t. -» «> as K -» <*> and ijr(t ■ ) -» 0 as K <». This con- 
■K- K 

tradicts the assumption that Be A,(-A) - -Ee A, (A) § o. This proves 
Proposition 5 . 5 . 


Example 5.1 . Let Lx = x + a^x + a^x, h = 1, = 2, h = 1, 

. 2 2 4 - 

a X = 2 lo S 2 , a 0 = (log 2) + r . Then A,(A) = (-log 2 Z wx], 
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and 

= exp[t(log 2 - tti) ] 
is a solution of L*x = 0 satisfying 
2i|r(t) + i|r(t+l) = 0 

and for this system ^(A,B 0 +e -A ^B^) is not normal,, 

Example 5 0 2 „ Using the same notation as Example 5»1, consider the 
control system 

L*x = u(t) + 2u(t-l) c 

Then 

t(t) = exp(in - log 2)t 
is a solution of Lx = 0 and 
ty(t) + 2ijr(t+l) = 0, 

and for this system j^(A, is not normal 0 
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6. Synthesis for the Special Time Optimal Control Problem 

Neustadt 1 s method of synthesis [21] can he extended to cover 
problems pg, P°. Some rather restrictive assumptions are re- 

quired for problems P-^ and Pg • Xt is assumed m our discussion 
in this section that x = 0 (x^ is the "target”). The 
development will be carried out only for problem P , but if the 
arguments are suitably adapted problems Pg and P^. can also be 
treated. The validity of Neustadt’ s approach depends on the follow- 
ing condition for problem p: If [u,u] is an extremal control 

for problem P satisfying x(t; x Q ,u) = 0, then [u,t] is an 
optimal solution to problem P. Neustadt [21] assumed that the 
system £f(A,B) was normal so that the above condition turns out 
to be satisfied by the sufficient condition in [11, PS. 7^] . For 
the problems we are studying, however, the optimal control {u, t] 


may be unique where all three of the systems B^) , 

are normal and yet i^(t) can fail to 
be expanding (see Example 7.1) so that the analogous sufficient 
condition for problem P^ could fail. 

Recalling the definition of z(t,ri) m Equation (^.5), we can 
obtain the following proposition. 


Proposition 6.1 . Let the following conditions be satisfied; 
her (B q ) C ker (B^, y(A,B Q ), V{k,P^e~ K \ ± ) , V{k,\) are 

normal, and (h 2) . Let S = [t] e R n j <t|,Xq> < 0} . If the optimal 
control (u,t} exists for problem P^, then it has the form 



(4.3) » Any vector 7 e S which maximizes the time t for which 


<T b r l) > = “ <T b x 0 > W he used m (4.3) to obtain the optimal 
control [u,t} = (u(»,t,7), t} . Conversely, if t) defines the 
optimal control {u,t} by means of (4.3 ), then it maximizes the 
above time t. 

Proof. Note that g(t, 7) defined m (M) can be written m the 
form (4.7) if t ^ h, and if 0 S t S h we get 

(6»1) g(t,n) = / he“ AS B J + he" As B |ds. 

0 x 

Hence (4.7), (6.1), and Corollary 4.1 imply that (t,Tj) >0 so 
that t | — »g(t,T}) t l 0 is strictly increasing. The function 
(M) H > g(M)> t § o, 7 e s is continuous. For 7 e R n , 7 ^ 0 
we have that 

(6.2) <7,z(t,7)> > <7,20, x e ^(t), x ^ z(t,7), 

by the normality assumptions in the proposition. Define 
f« [0,co) x S x R n -> K by the equation 

f 0M, x o ) s<n,z(t,n) + x Q >, 


and define 
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H 0 * tn « H 11 ! „ + 0 , <n,-x 0 > = <n,y». 

The set H Q is convex. Observe that 
(6.3) f(0,ii,x 0 ) <0, n e S 


whereas 

(6.4) f(t,ii,x 0 ) > 0, T) eS\H 0 . 

The assumptions of the proposition imply that ^(t) expanding. 

Hence Theorem 4.1 and relation (6.2) assure us that 

(6.5) *(M,x 0 ) = 0 

implies that t = t if 7i e H Q . Hence using (6.3), (6.4) and the 
last remark it is seen that (6.5) defines t implicitly as a func- 
tion of t], for n e S. We denote the function so defined by F. 

Then F is continuous and F(r|) » 7] e and t > f(t]), r) e S\Hq. 

The purpose of the observation m the above proposition is to obtain 

a method for finding a vector T] which can be used m (4.3) to determine 

1 

the optimal control. It is easy to see that g is a C function on 
([0 a>)\(h}) X s by direct computation in formulas (4.7) and (6.1) 
using standard results on the differentiation of Lehesgue integrals 
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involving parameters [20, pp. 216-217] and the normality hypotheses 
of Proposition 6.1. Hence if t] e S is such that F(t]) j h, and 

-3! 4 0, then the implicit function theorem tells us that 

F is continuously differentiable on a neighborhood of rj. Using 
the fact that under the assumptions of Proposition 6.1 i^(t) is 

expanding (so that the sufficient condition, Theorem 4.1, applies 
to P 1 ) and the above remarks, the gradient technique for determin- 
ing the maximum of P on S can be applied to Problem P^„ We do 
not carry out the details here, but refer the reader to Weustadt's 
paper [ 21] „ 
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7 . Examples o 

In this section we solve some examples which illustrate the 

strange behavior of solutions to problems of type P.^ P g , P°„ 'All 

of the examples are two dimensional. Since we would like as much 

as possible to avoid using superscripts and subscripts, we shall 

1 2 

agree m this section that (x,y) = (x ,x )„ 

Example 7 0 1 0 The system equations are 


(7.1) x = y 

y = u(t) + u(t-l) . 

, ®q = ^ = (^) } 4 = 1« Here we consider 

p with boundary conditions, 

(7.2) x(t x ; (x 0 ,y Q ),u) = y(t x ; (x 0 ,y Q ),u) = o. 

2 

It is not difficult to see that given any ( x 0 > y o) 6 R 4here 
is an admissible {u,t..} satisfying (7.2). Hence there is 
(Theorem 5.2) an optimal solution to problem p . Proposition 5.4 
and Theorem 5.1 assure us that the optimal control {u,t} is unique 
if t S 1 and if 0 § t < 1, {u,t} is unique where it is effective, 
i.e,, on [-l,t-l] and [0,t]. The necessary condition (2.1) when 
applied to this problem yields 


111113 A= fo o) 

a problem of type 
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(7.3) 


u(t) = 


sgn [t (t+1)] 


undetermined 


sgn [t (t)] 


.1 S t S t - 1 
t - 1 < t < 0 
0 ^ t s t 


if 0 S t s 1, and if t > 1 } then 




sgn [i(f 2 (t+l)] , 

-1 s t < 0 

(7.3 T ) 

u(t) = 

sgn [i|r 2 (t) + \jr 2 (t+l)]. 

ost<t - 1 



sgn [* 2 (t)] , 

t - 1 1 t 1 t 

where if = 


is a nontrivial solution 

of the adjoint equa- 

0 

tion if = 

2 

-iJfA. Hence if (t) = pt + 6 where 

2 

1 ]/ (t) is not 


identically zero., Along with the optimal control {u^t} we con- 
sider the effective optimal control (v,t} where 


(7.4) 


v(t) = u(t) + u(t-l), 


With problem for system (7.1) and boundary conditions (j.2) we 

consider the auxiliary problem P with system ^(A^B^) only with 
the restraint set changed to [-2^2], The synthesis for this 
problem except for an obvious scaling factor of 2 (i.e., the 

switching curve is x = _y fk } y g 0 and x = y /4, y § 0) is 
described in [22] „ if {w,T) is the optimal solution to the 
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auxiliary problem P and if w is expressible in the form 
u(t) + u(t-l) with {u^t} admissible m P then {u^t} is the 
optimal solution of P_^„ Thus P^ can be considered solved if 
0 i t ^ 1„ Figure 1 shows the reachable set i^(l) and the 
synthesis in case (x 0 ,y Q ) e 


Figure 1. 


Thus we now assume that (x^y^) / iS^(l) so that t > 1. Here 
the situation is a good deal more complicated since the above w 
is no longer expressible m the required form. It is noted from 
(7o5 J ) and (7=4) that the effective optimal control has v(t) talcing 
only the values m the set [-2 ; 0,2} ; 0 § t < t„ For brevity let 
us denote the optimal trajectory issuing from (x^y^) by (x,y) „ 
Then (x(t)^y(t)) can reach (0,0) only along one of the two 
curves 


S + : x = y 2 /4, y ^ 0, 
S l x = -y 2 /4_, y ^ 0„ 


If p = 0, then 6^0 and from ('{<,3') we see u*(t) = sgn (S), i 0 e.. 



there is no switching* Hence 8 > 0 implies (vV € s + 311(1 
8 < 0’ implies (x 0 ,y Q ) e S„o Conversely, (x Q ,y 0 ) s S + U S_ implies 
p a 0, If (x 0 ,y Q ) ft S + U S_, then p / 0. It is not difficult 
to show that p > 0 or p < 0 accordingly as (x Q} y Q ) is to the 
right of S, U S or to the left of U S_» Let -8/p he de- 
noted hy p* If one finds v(t) = u(t) + u(t-l) using (7.3'), 
then it is clear that "both v and u will he known completely if 
the disposition of the points - ^ + p, P, ^ + P relative to 
[0,t] can he discovered* Now the boundary conditions (7.2) impose 
additional conditions on t and p. In fact with u = u and 
t^ = t (7*2) reduces to 

t _ 

(7.5) x = f sv(s)ds 

0 

t 

y Q =“/ T(s)ds. 

o 

By a systematic and laborious enumeration of the possible positions 

11 — 
of - £ + p, p, — + p relative to [0,t] it can he shown that 

(7*5') and (7*5) uniquely determine t and p as functions of 

/ 

(x Q ,y 0 ) / S + U S * In principle at least the determination of 
t(x 0 ,y Q ) and P(x 0 ,y 0 ) represents no difficulty, so we shall only 
describe the results* However, it must he pointed out that when 
the possibilities are exhausted our calculations revealed the 
following^ There corresponds to each (x ,y Q ) e R \ 1^(1) exactly 
one extremal control satisfying the boundary conditions (7*2). 
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Hence an extremal control satisfying (7.2) must He time optimal, 
(it will "be seen momentarily that J^^(t) is not expanding, so 
Theorem 4.1 does not apply.) 

Let D (respectively, D ) denote the open region to the 
R Jj 

right (respectively, left) of S + U S_. Sets D_^, i = 1,2, „..,7 
are defined by the following relations ° 


h = f Vo> 


e D r 1 x 0 2 “ T " T } > 


= 


( (V y o ] 


2 2 

e D r | -2 g y 0 < o, £ - -g- - ~~ 1 x 0 g \ - + -g }, 


1 y 0 y o 


n, = 


^V y o } 


1 y o y o 


Vr 


e D rI y 0 - O ^““^“"^ <X O-I-| y O + “^ }U 


i y o , y o 


3 y 0 A 


{( X 0 ,y 0> e d r 1 y o - °' 2 " + “8- - x o - 2 “ “2 " "S’ 


e 4 = 


{(x 0 , y 0 ) 


e DR ! y o,- 2 ^, Xo g T-^^ 


y o y o 


D_ — 


f(x 0 , y 0 ) e D r I y 0 g °> ~ “I + “? - X 0 - S’ " 2y 0 + “4 


y, 


o 


3 y o y o 
X 0-2--^"-F^ 


Dr- = 


{ W 


£( V y o } 


e V 0 


3 3 


y, 


o 


3 y o 


-^ 0 -^2 2 ,y o 8 _ 0 2 4 ; 


e V y o s x o s I - T } > 


D„ = 


» x 0' y 0 ) 


€ d r! y o s °- x 0 s I - y 1 u 


l(> y 0 ) e B b | y 0 S 0, x 0 S | - 2y 0 + } 


We define D , i = 1,2, c<1< ,,7 hy symmetry through the origin, i.e.. 


Then 


k9 

= -(x 0 ,y Q ) e 1 = l- } 2 } ° •> ° ,7 ° 

■ 7 i r t i 2 

U dJ U I UdJ U [S + U s ] is E, and ^( 1 ) = cl (D 1 ) U 

cl (D 1 ), where cl (E) denotes the closure of E„ The regions 
D i# S + , S , 7 are depicted in Figure 2. 


Figure 2, 


The following formulas obtain for t and (3! 


t(x 0 ,y 0 ) - 


y 0 + v 2 y 0 + 8x 0 

2 > (V y o 5 6 D 1 

x o i y o i , , „ 

" " “ + 5 ’ X ° ,y ° ^ 2 


1 + *4 + J^ 4x o + ^n/ 2 + 2y 0 - 2 > (x O^O } e D 3 


" -? * ( V y cP e B k 


y * 

-? + 2 + T£ V3Yq + + &x n , (x, v y n ) 6 D c 


o * v cv'cr 5 


+ 5 - I + Y V h) > (x 0> y 0 } e D 6 


^0 

-2 


+ \/l + 2(x Q + y 2 /4) , (x 0 ,y Q ) € D ? , 
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Jr 


> <V y o ) e s - 


t(x o>V " 


y 


o 


> (v y o> 6 V 


y + 2t 


> ^V y 0 } e D l u D 7 


y + 4t - 2 
J 0 


> (Vo> e “a 


p ( x 0 ,y 0> " 


y o + 

8 


> <VV e D 3 


y Q + 2t 

2 ? 


( V y 0 } e D 4 


y 0 + 2t + 2 

6 ' ^ X 0 ,y 0^ 6 D 5 


y + 4t - 2 

5 j (*o ,J c) e d 6' 


It is noted that if D fl f 0 for some i ; k = 1 } 2 } ... } 'J } then 

there is still no ambiguity in the formulas for t(x Q ,y ) and 

P(x Q; y^) » In order to complete the definition of t and g on all 
2 

of R we merely take advantage of the symmetry m the problem to 
observe that t(x 0 ,y Q ) * t(-x 0 ,-y Q ) and p(x o ,y o ) = p(-x o ,-y Q ) if 
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(x^.v.) e D . We note that t is not continuous at points on 
D O Jj 

S + U S and on n cl (D^ ) „ However ^ at every other point of 

D hoth t and p are continuous. 

K 

How to see the nature of the optimal trajectories we describe 


the optimal effective control v(t) = u(t) + u(t-l) if the initial 
data (x 0 ,y Q ) e i = 1,2 J . 00 ^7 0 We use V 1 to denote the 
optimal effective control defined on [ 0 ? t (x^y^) ] if ( x q^q) e 
i - 1,2^ 000 ^7 0 Of course, if (x Q ,y Q ) e D then the optimal 
effective control is -v^ i = 1,2,..., 7 . The formulas for v 
are as follows l 


v l (t) = 


_2 

+2 


o g t g p 
p<tst 


r 0 


> 



+2 


} 


0 § t g t - 1 
t - 1 < t g p , 

p<tst 


( 1 ) 


“-2 , 0 g t g - | + p 

0 , -£+P<tgt-l 
-2 , t - 1 < t g p 

+2 , p < t g 1 

0 , 1 < t g j + p 

+2 , ^ + p < t S t 
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0 g t S p 
P<t « t 


■v 5 (t) 


"-2 , o^tg-i+p 

0, -i+p<tgp 
+2 f p < t =S 1 

0, l<tsJ+P 

+2 } i+p<tst 



-2 , 0 S t g - p 

0, - j+p<ts t - 1 

-2, t - 1 < t £ p 

o , p < t g ^ + p 

+2 } j+p<ta 


v T (t) 


-2 , o g t g - i + p 
o , -i+P<tg|+p 
+2 , ^ + P < t S t 


If CvV € D +i u e +2 u d +4 u d +6 u \r then the optimal 

trajectory, (x(t),y(t)), beginning at (x^y^) can be described 
in a simple geometric fashion. If (Xq^q) € cl 0^ U D 1 ) = &^( 1 ) } 
then this description is given in Figure 1. Evidently, if 

s V then v^(t) switches from 0 to +2 as (x(t),y(t)) 
crosses S + . Moreover, m all cases where (x^, y^ ) e D R the last 
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switch occurs as (x(t),y(t)) crosses S + „ Wo^ let C 2 ^ = 

{(x,y)| x = ^ ^ , >2 ^ y g 0) (note C 2± is a segment of 

Bd (i^(l))). Then if (x Q ,'y 0 ) s D^, the point (x(t),y(t)) coasts 
to C 21 and then the synthesis for i^(l) obtains (Figure 1). Let 
curves C^g, ^63 ^- e - ine ^- as follows” 


c 6i = x = I - § y + ^ > - 1 - y - 

2 

c 62 = { ^ y ) i x = I - I - % ^ -l s y s l), 

2 

c 65 = C(x,y)! x = - § + \ > -i^ygo}. 


If (x ,y ) c Dg, then the first, second, and third switches of vg(t) 

take place as (x(t),y(t)) crosses Cs , i = 1,2,3 respectively <> 

2 01 

Finally, define = {(x,y)| x = 2j_ - y, yg -1) . If {^Yq) « D , 

then the first switch of v (t) happens when (x(t),y(t)) crosses C 
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If (x 0 ,y Q ) slgUD^UDgUD^ then hy use of symmetry the optimal 
trajectories are similarly described using curves C = -C. . The 
synthesis for (x Q ,y 0 ) sDgUD^UDgUD^ is shown m Figure 3 = 


Figure 3. 


For (xQ,y Q ) s E, U D_ the set of "first switching points" do not 
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lie on a curve and the situation is too complex to describe geo- 
metrically. Some typical optimal trajectories are given m Figure 4 
for (x 0 , y<J ) s D +J U D +5 . 


Figure 4. 

It is interesting to note that some of the optimal trajectories 
initiating in or Dg can come to rest on the x-axis for a 

positive time duration before continuing on to the origin. Tra- 
jectory A m Figure 4 shows an instance of this, but this is not 
typical. 

In this example, ^(A,B q ), and i^A^+e"^) are 

all normal (and all proper) and yet ^(t) is not expanding al- 
though &^(t) is increasing. The boundary of l^(t) for a few 
v 3 J_u.es of t is sketched m Figure 5 © 


Figure 5. 


This figure clearly shows ^(t) is not expanding. 
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An example of a problem of the form 


is now considered. 


Example 7.2 . The example considered here is exactly the same as 
that treated in Example J.2 except that here we impose the con- 
straints 


( 7 = 6 ) 



U = 0 o 
t l 


We give only a brief 
able set i^°(t) is 
Sf( A,B 0 +e-\) o The 


discussion of the solution to P°. The reach- 

3 

the same as i?(t-l) for system 

control system SS(A f - B Q +e_ ^B^) is given by 


( 7 . 7 ) 


x = y - u(t) 
y = 2u(t). 


2 

Wow given (x^y^) € R there is an admissible control {u,t-^} for 
P with system (7.7) such that the response of (7.7) to this con- 
trol satisfies 


x(t x ; (x 0 ,y 0 ),u) = y(t x ; (x 0 ,y 0 ),u) = o. 


Hence the same is true of (7.1) with {u^t^+l} 
This assures us that an optimal control (u,t} 
unique (Theorem 5.2, Proposition 5.5). We note 
m (7=7) then we obtain the curve 


admissible in P„ 


for P, exists and is 


that if u(t) = 1 
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2 

O • yr ~ y _ y y ^ 0 

b + . x — jj- - 2 > -y - 

and if u(t) = -1 we obtain 

2 

s : x = - TP - I J yso. 

Figure 6 illustrates the synthesis for u using the auxiliary tra- 
jectories from (7oT)o 


Figure 6» 

We let D_ (respectively D ) denote the open region to the 
E L 

right (respectively, left) of S + U S „ For this problem the 
regions D , i = 1,2, . 5 are as follows^ 

_2 

D i “ l<Vo> s d rI x o 4 S' - y o - -§ 1 - 

2 2 

D s - l(Vo> £ “e 1 5 - y o - T s x o “ 2 - I y o + y 

2 

y o y o 

and x Q S 2 ^ ^ } , 
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D = 


“VV sD eI a -^-T Sx o s2 -ho + ^l 


3)i. = 


«V y o> 6 V y o s 2 “* x o ‘ 2 - T -V - T > u 

2 

^V y o } e d rI - 2 2 - I y o + T - x o - 2 


y o y o 

" — 


} , 


Vr 


y^ y. 


D K - {(x^.y^) e D_l x ^ 2 _ 2 . y + _,£ x £ 2 2 . _ _£ ] 

5 0’ J 0 J R 1 0 2 J 0 T+ 7 0 _ 2 h 1 


The sets D i = 1,2, .. „,5 a^e defined by symmetry as in 
Example 7.1. We have 




U [S + U 3J = B 


The regions D +i , i = 1,2,..., 5 are shown in Figure 7. 


Figure 7 . 

Using the boundary conditions (7.2) and the maximum principal 

for one can show 

5 
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i s'H g o 
o < t g x 

X < t < t - 1 3 

t - 1 g t g t 

where 

^VV " 1 + “2 + 3^0+ !|y o + 8x o ’ 

W > _ y 0 . t - 1 

MVV *tj: 2" > 

if (x^,Yq) e D^o By symmetry we have 

= ^ JC 0'- y 0 ) ' = *' (_x o’" y o' 1 

if (x 0 ^y Q ) € D . The optimal effective control {v,t} for 
(x 0 ,y Q ) e is denoted by v, l = +l,+2, „ , .,+5. Evidently, 
v^ - -v i = 1,2, . „ 0 ,5„ The following formulas for v are 



obtained 
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-l, o £ t < x 

+1, X £ t < t - 1 

V (t) = 0, t - 1 £ t < 1 

-1, 1 £ t < X + 1 

+ l y \ + l S t S t 



0 £ t < X 
X £ t < 1 

1 £ t < t - 1 
t-lstcx+l 
X + 1 £ t £ t 



0 £ t < X 

X £ t < 1 

X £ "fc < X + 1 

\+l£t<t-X 
t - X £ t £ t 



0 £ t < 1 

1 £ t < X 

X £ t < t - 1 
t-l£t<X+l 
X + 1 £ t £ t 



6o 


-l 


} 


- 2 , 
0 , 
+ 2 , 


0 g t < 1 

1 g t < X 

X £ t < X + 1 
X+l£t<t-l 
t - 1 g t g t 


Figure 8 shows some typical optimal trajectories for P° with 
(x ,y ) e D +1 , 1 = 2, ooo,5« Figure 9 illustrates some additional 
curious phenomena for this problem e 


Figure 8» 


Figure 9* 


For example if (x^y^) = (-2,2) e D then the optimal trajectory 
to the origin is simply the arc pO of the curve x = -y /2 
connecting (-2,2) to (0,0) (Figure 9). > However, a subarc pq_ 
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of arc pO is contained m Dg. Thus if one starts at a point n 
on the subarc p<p, then the optimal trajectory does not follows arc 
nO to the origin, hut will go off on a rather pathological trajectory 
(curve x in Figure 9), finally coming to the origin on an arc rO 
of the curve x = y / 2 C Figure 9 also depicts what can happen when 


x. 


2 2 
= +y / 2 „ For example, starting at point i on x = y /2 the 


0 -"O' 

optimal trajectory is the curve cr 0 Note that a in Figure 9 hits 
2 

0 at time t - — bounces down and then swings back to hit the 

origin at time "fc = 2„ Other variations of ijhis type of behavior 

can also occur because of the boundary conditions on the controls m P°. 

The next two examples demonstrate what can happen in problems 
which are not "normal" and where ker (B q ) and ker (B^) are 
complementary spaces (see section 2). For these examples the 
attainable sets at time t can be determined without difficulty, 
enabling one to make a judicious choice (whenever there is more than 
one support hyperplane at the boundary point) of an outward normal 
for use directly m the maximum principle „ 


Example 7 <>5 ^ This is an example of the form P^. The system equa- 
tions are 


(7.8) x = u(t), y = u(t-l) 

with boundary conditions the same as in equation (7=2) . For system 
(7.8) the domain ^ of null controllability (here U = [-1,1]) 



62 


turns out to be 



(ft 


■0> y 0> 


€ R 


IVV 4 a >' 


Thus the problem P has a solution only if e ^q° 

the other hand if (x 0 ,y Q ) € Q}\, then Theorem 5«2 assures us that 
problem. has a solution. The attainable set at time t turns 

out to be (x 0> y Q ) + ^(t) which we denote by S^ t {x Q} j Q ) and 
this can be explicitly computed^ 


= ^ x » y ) e r2 

Figure 10 shows (x^y ©) , 1 

l 


|x- x ol g *, |y-y Q [ g I x -y| g 2 )° 

1,2, for t < 1 < t g . 


Figure 10. 

Taking advantage of the simple geometric structure of ^^(t) 
one finds that t(x 0 ,y Q ) = max Ux Q | , |y Q |'}, (x 0 ,y Q ) e 2>^. Thus 
an admissible control [u,t}, t - t(x ,y Q ) satisfying the boundary 
conditions (7»2) is a time optimal solution. The maximum principle 
for this problem says that if rj = (t) ^ 0 is a vector which is 
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an outward normal to a support hyperplane for J32~(x 0 ,y 0 ) passing 
through (0,0) and if t < 1, then 


(7.9) u ( t ) 


2 - 

sgn -ist^t-i 

undetermined, t - 1 < t < 0 
sgn [r) 1 ], OStit 


and if t ^ 1, then 


(7« 9' ) u(t) 


sgn [n 2 ], -1 S t < 0 

sgn [tj^+ti 2 ], 0 ^ t < t - 1 

sgn [r) 1 ], t-Htlt 


Let us consider some of the possibilities. Suppose —(x 0 ,y Q ) is 

on the line y = x - 2 and y Q > -x QO Figure 11(a) shows how 

1 2 

J3C_(x ,y ) is positioned at (0,0) and we see that 1 = (p ,t) ) 


V J 0 

" 1 2 
can be chosen so that i] < 0 < t| and 


1 2 

r\ ] > t) . Using this T) 


m (7o9’ ) one obtains 


u(t) 


+1, -1 S t < 0 

-1, ostst 


if t S 1, with an obvious modification using (7*9) if t < 1. 
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Figure 11. 


If (vV € 

then J3<_(x 0 ,y 0 ) 

w 


Si\ ana J(> > - Xq , Xq . 2 < y Q < X Q , X Q > 1, 
is positioned at the origin as shown m 


Figure 11(b), where t = x Q > 1. Hence ti = (h ,0), q 1 < 0 and 
(7=9' ) gives no information on the interval [-1,0) but (7.9') 
does specify u(t) = -1, 0 I t ^ t. In this situation it turns out 
that any u such that (u, t] is admissible in satisfying 
u(t) = -1, 0 g t § t, and which drives (x^y^) to (x^-1,Xq- 1) 
at time t = 1, turns out to be optimal. Let (x,y) denote a 
response initiating at ( x 0 ,y 0 ) to a control {u,t} of the above 
form. Then we see that x(t) = y(t), 1 g t § X and 


(7=10) |y(t)J/|i(t)| § 1 

for 0 g t g t. On the other hand if > -x , x - 2 < y < x . 

J 0 0*0 J 0 0 * 

and Xq g l(t g 1), then (7=10) 13 all that is required of the ad- 
missible trajectory (x,y) ns long as the boundary conditions (7.2) 


are satisfied. 
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Suppose now that x Q = y^ and y^ > 0„ 
t = y Q and 


Then we find that 


u(t) = -1 -1 g t g t ; 


where {u^J is the optimal control. 

If y Q = -x Q and y Q < 0,, then the optimal control {u,t} is 
given by 


u(t) = 


+1 

_1 


-1 g t g t - 1 
0 g t g t 


where t = jy Q | ^ 1, 

Using similar techniques one obtains optimal controls {u^'t} for 
all (x 0 ,y Q ) lying in with y^ > -x Q , By taking advantage of 

the symmetry with respect to the origin an optimal control can then 
be determined for (x^y^) in the remainder of £2^. 

Figure 12 illustrates the typical situations. In this figure 
heavy lines indicate pieces of optimal trajectories when {u^t} is 
unique^ and the broken lines indicate segments of optimal trajectories 
where the uniqueness of the optimal control does not obtain,. 

In this problem j^(t) is increasing but not expanding. 
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Figure 12 o 

Example T a b a In this example we look briefly at the same situation 
as m Example 7.3 except we change to a problem of type where 

v = 0. how the domain of null controllability (with U = [-1, 1] ) 
is 

^(°) = 6r2 | iV y o^ " 1} > 

and the attainable set at time t which we again denote by 
-Q^_(x 0? y 0 ) is equal to (x 0 ,y 0 ) + ^ 2 (t). It is easily shown that 

-^(V y 0^ = 6 ^ i X “ X ol - l y_y of - t - ^ l x “ y l - ^ 

for t § 1 and 

= [(x ' y) e r2 I x - x o ! - t ’ y = y o } 

for 0 < t < 1. These sets are shown in Figure 13° 
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Figure 13. 

Again, if one takes advantage of the simple geometry present m 
the problem, then the "minimum time" t is determined to he 


t <V 3 r o ) = 


' 1 + h 0 t , Vo le V , ’ r o /0 


x 


O' 


( V y 0 } e ^ 0) > 


O' 


0 


which is discontinuous on 3^(0) at every point on the line 


|x 0 l < !, y Q = °}. If (x 0 ,7 0 ) e ^(0), then 341 


{(x 0 ,y 0 ) « h 

optimal control (u, t] exists for problem and t = t(x^,y ) 

(Theorem 5»2). The necessary conditions for this example are the 
same as in Example 7.3 (equations ( 7 . 9 ) and (7. 9')) except the con- 
dition on u(t), -1 ^ t < 0 is deleted. To solve this problem one 
considers (as in Example 7,3) the possible tj = (ri ,7 ) which are 

normal to support hyperplanes for j^_(x y ) through (0,0) and 

t 0 0 

makes an appropriate choice when there is more than one candidate. 
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We now consider some of the eases. If y^ a 1 (i.e., t S 2.), 

then (f.9 1 ) yields 

(7.11) u(t) = -1, OStSt-1. 

If m addition to y Q § 1 we have y Q = x Q - 1, then u(t) = -1, 

0 S t S t and the optimal control [u ; t} is unique. On the other 
hand if y Q § 1 and y Q = x q + 1, then m addition to (7.11) we 
find that u(t) = +1^ t - 1 < t %. t } and again the optimal control 
{u,t} is unique. Wow if y Q S l and x Q - 1 < y < x + 1, then 
any admissible control u will he optimal as long as it satisfies 
(7.11) and is defined on [t-l^t] so that the boundary conditions 
(7.2) are satisfied. The cases that we have Just discussed are 
shown m Figure 14 (where again non-unique segments of optimal tra- 
jectories are denoted by broken lines) by the trajectories initiating 
at points p^* p £ ; and Pj^ respectively. We note that m 
many cases the optimal trajectories contain subarcs which lie out- 
side the domain of null controllability. 


Figure 14. 
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If y Q - 0, then one can show that 

u(t) = -1, 0 § t § t if 1 £ Xq > 0 


and 


u(t) = +1, 0 g t ^ t if -1 ^ x Q < 0, 

so that the optimal control is also unique and the corresponding 
trajectories are very simple,, Finally, we consider one other typical 
situation when optimal controls u are not unique . Suppose 
0 < y Q < 1 and x Q - 1 < y Q < The necessary conditions still 

give (7„11), hut m this case any admissible u satisfying (7„ll) 
and the boundary conditions (7.2) at time t = 1 + y^ < 2 is 
optimal. For example the trajectory issuing from point p g in 
Figure 15 shows one of the many optimal trajectories starting at 
this point at time 0. This trajectory 'passes through at time 

t - 1, arrives at r^ at time 1, passes through s^ at some 
time t, 1 < t < t, and finally arrives at 0 at time t. 

Other optimal trajectories are also illustrated in Figure 15. 

In this figure once again heavy lines denote pieces of optimal tra- 
jectories where the optimal control is unique, while along broken 
lines the optimal control is not unique. 

In this example ^g(t) turns out to be increasing but not 


expanding 



TO 


Figure 15 o 


Finally, it is noted that if we consider system (7=8) with a 
problem of type then the domain of null controllability is 

merely the straight line px, This problem is easily solved and 
some optimal trajectories for this problem are depicted m 
Figure l6 0 


Figure 1 6 . 
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8. Delayed Control Problems and Dynamic Programming 

Consider once again Example 7.4 above. If we consider the 
optimal trajectory emanating from m Figure 14, we notice that 
this trajectory has suharcs which are not optimal. Thus the 
principle of optimality m its usual form. [19] does not hold here. 
This is not too surprising since this principle fails even m 
ordinary control problems with time dependent restraint sets U(t) 
if one interprets "state" to mean x(t) instead of (t,x(t)) 

(cf„ [16]) . However } m the problems we are studying this difficulty 
is more serious. 

We also observe that m Examples 7-1 and 7°2 the principle of 
optimality m its usual sense fails to be true. On the basis of 
this experience one expects the failure of this principle of 
optimality to be an intrinsic property of optimization problems in- 
volving systems of the form B ,B^) and not just a peculiar 

property redounding from the special boundary conditions in 
Examples 7.2 and 7.4 or the particular criterion for optimality. 

Hence one anticipates serious obstacles to obtaining results for 
problems involving ^(A^B^B^) using dynamic programming. 
Nonetheless^ for certain special performance indices we axe able to 
adapt the methods of dynamic programming to problems governed by 
systems -S^(AjBq ? B.^), even though it is easy to construct examples 
showing that the standard principle of optimality is also invalid 
for these problems. 

The remarks below are valid for time varying systems even though 
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we shall, m keeping with our practice in this paper, restrict our 
presentation to the case of constant coefficients. 

2 

Let a'. R n R and Li RXR -» R he given C functions. 

Suppose U C R m is given and t^E is fixed. For t Q < ~ h 

we shall denote hy H the problem of minimizing 

t l 

J(u,° t 0 ,x Q ) = + / L(t,u(t))dt 

t o 

over the class of j^hadmissible controls u where x( * , tQ,x^,u) 
is the solution of (v is a given fixed function 

throughout) subject to x(t Q ) = x^. 

Remark 8.1. We shall consider only the free endpoint problem* 
problems with restricted endpoints x(t^) e Z C R n require the 
usual modifications [3, 19] ® 

2 

An easy calculation shows that the response to .S^C A, Bq,B^) 
satisfies 


x(t x ; t 0 ,x Q ,u) = x(t ± ; t 0 ,Tx Q ,u) 

/S 

whenever t^ ^ t^ - h, where x is the solution to 
(8.1) x(t) = Ax(t) + 2(t,t 1 )u(t) t e 

subject to x(t^) = KCq with Tx^ = x^ + / e” ^ +h ^ A B^v^(|)d| and 
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B 


0 


B 0 + 


t € 

t < t - h 


We shall denote by A the problem of minimizing 

h 

J(n; t 0 ,3D 0 ) s cfxftp) + / L(t,u(t))di 

t 

0 

over all bounded measurable controls uS [t^, t n ] -> U where 
t Q < and x(*° t 0 jX 0 ,u) is the solution, to (8.1) subject to 

A A 

x(t Q ) = x . Note that the payoff J ( J ) depends only on 

x(t-^) ( x(t^)) and not on x(t) ( x(t)) for t < 

A 

Since J (u; t Q ,x ) - «7(u,° t ,33^) for every t < - h and 

Xq e B , we see that the problems n and H are equivalent when- 
ever t s - h. That is, if, for given initial data (t^x^) 

A 

with t S t x - h, u is optimal for H, then u ; extended to 

[t 0 »h,t ] by taking = v , is optimal for IT with initial data 

-1 ^ — 

(t^, I x q)« Conversely, if u is optimal for H with initial data 
(t^,x^), I - h, then u restricted to [t ,t^] is optimal for 

A 

h with initial data (t , Tx Q ) . 

A 

Applying the methods of dynamic programming to the problem II 
we obtain the Hamilton-Jacobi equation [5, 19] 


S' (t,z) + min (L(t,w) + $ (t,z)f (t, z,w)} = 0 
well 


( 8 . 2 ) 
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H a s* 

for t < t„ and z e R } where ^(t^z) = inf J(u‘ t^z) and 

" L u 

f (t^ z } w) = Az + n^t^w. Solving (8.2) with data S(t^z) = cr(z), 
one obtains $(t o ^x Q ) for t Q < t-^x^ e R n . Since for 

t Q g ^ - h we have = $(t Q; Tx o )j where $(t,z) = inf J(uJ t, z), 

one thus has the optimal payoff for problem H. It should be noted that 
although (8.2) is valid for t < t-^ one has 4>(t ; z) = <5 (t^ Tz) only for 

i /v 

t § t^ - h. In case v Q = 0, one has Tz = z and $(t,z) = $(t^ z) 
for all t < t-^. 

A 

Let us now consider a special case of the problems K^H for 

which (8.2) can be solved using known techniques. Denote by H 
A 

and H respectively the problems H and H for quadratic payoffs 

a(x) = xSXj L(s,u) = uR(s)u where U = R m . We assume that S e 

is symmetric positive semi-defmite and R(s) e £ is symmetric 

mm 

positive definite for s e R . Application of known results to the 

A 

problem H yields the optimal (feedback) control 
(8.5) u(t) = -R“ 1 (t)fl*(t J( t 1 )G(t)x(t) 


for t e [t^t^J w ^ ere G satisfies the matrix Riccati equation 


(8.4) G(t) + G(t)A + A*G(t) - G(t)a(t,t 1 )R“ 1 (t)^(t^t 1 )G(t) = 0 


for t e [to^tq] with boundary condition G(t 1 ) = S. Rote that 

(8.3) gives a feedback solution for the problem H which can be 

used to solve the problem II m the following manner. Given 

y. 
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(t ? Xq), t Q g - h, as initial data for the problem II one solves 

S\ . 

the problem. *wath initial data obtaining a feedback 

of the form (8.3). Next one uses this in (8.1) to find the optimal 
xj i.e«, one solves 


(8.5) x(t) = {A-I2(t^t 1 )E" 1 (t)fi*(t_ ) t 1 )G(t)}x(t) 


for t f [t ,tp with data x(t^) = Tx^. Using this together with 
(8.3) gives the optimal open loop control for H . 

This control can then be used m B^) with x(t Q ) - x q 

and u^, = v Q to find the optimal trajectory for problem This 

latter step is not necessary to find the optimal value of the payoff 
for n , since knowledge of x and u yields J(u° t^x^) at once 
from 


J(u; t 0 ,x Q ) - J(u; t 0 ,Tx 0 ) = x(t 1 )Sx(t 1 ) + / u(t)R(t)u(t)dt. 

t Q 


We note that in (8.1) and the performance index j(u* t^x^) we 
could make the change of variable y = e x, and then system (8.1) 

o 

takes the form y = Q(t,t,)u(t). If one carries out these sub- 
stitutionSj then the corresponding Riecati equation will have the 
simple form G _ GC(t,t 1 )G = 0 which can often be solved by a 
quadrature (see [25^ p. 227]). 


Remark 8 .2. It is not difficult to give a rigorous derivation 
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(including existence of the required solution to the Riccati equa- 
tion (8.4) on the entire interval [t ^t^]) of the above solution 

A 

to the problem. II using the maximum principle for and 

arguments s imi lar to those by Lee and Markus [19, sections 3.2 and 
3.33* 

Remark 8»3 » The above ideas can be applied to certain optimal con- 
trol problems where retardations occur in both the state and control 

variables. For the corresponding quadratic payoff problem II , one 

Q. 

can then use recent extensions of the Riccati theory [1, 8, 15, 17 

27 ] . 
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